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The Carnegie Foundation for the Advancement 
of Teaching is a nonprofit, operating foundation 
with a long tradition of developing and studying 
ways to improve teaching practice. The current 
president, Anthony S. Bryk, was one of the 
founders of the Consortium on Chicago School 
Research, a pioneering research alliance. He has 
led Carnegie’s work on networked improvement 
communities (NICs), which reflect his belief that 
it is not enough to study policies and programs 
as they exist today. Research in education should 
also tackle rapid engineering and testing of 
conditions for improvement. Bryk explained: 

Making progress in addressing 
educational problems reguires a 
commitment to a rapid prototyping 
process by which researchers and 
practitioners co-develop innovations, 
try them in schools and other learning 
contexts, and then refine and try 
them again. This new infrastructure 
demands an engineering orientation in 
which adaptability to local contexts is 
a direct object of study.' 

For the past three years, the Carnegie Foundation 
has initiated three different NICs. The first, 
Quantway, is addressing the high failure rate of 
students in developmental mathematics. Eight 
community colleges in three states are part 
of this network, as are several intermediary 
organizations whose work focuses on 
curriculum, faculty development, and student 
support. The second NIC, Statway, is also 
focused on community colleges. It involves 19 
colleges in five states and is a pathway to college 
statistics. The third NIC, Building a Teacher 
Effectiveness Network (BTEN), is working on 
teacher quality, specifically on developing and 
retaining teachers in the first three years of 
teaching. The BTEN network members include 
the Carnegie Foundation, the Institute for 
Healthcare Improvement (I H I), the American 
Federation of Teachers (AFT), New Visions for 
Public Schools (NVPS), the Austin Independent 
School District (AISD), and the Baltimore City 
Schools (BCS). Carnegie staff act as the primary 
facilitators of the work, guiding the improvement 
process. IHI and AFT also help facilitate the 
improvement work. AISD, BCS, and NVPS are 
sites that test ideas. 


History 

In 2009, the Aspen Institute convened 
representatives of the institutions that now 
make up BTEN. Initial meetings in fall 2009 
and early spring 201 0 resulted in four goals: 

(1) improve the entry of new teachers into the 
profession; (2) enhance the capacity of novices 
to learn to teach; (S) achieve measurable success 
in the retention of effective early-career teachers 
and the student learning gains assessed in their 
classrooms; and (4) build a learning community 
around this work. 2 

Since 2010, the Carnegie Foundation has 
facilitated the development of BTEN as a NIC, 
with a focus on organizing its work using the 
model of improvement science developed by the 

IHI. 

Nature of the Partnership 

Work in sites began in the 201 1-201 2 school 
year with a focus was on the quality of feedback 
that new teachers receive from people in their 
districts, especially principals. Carnegie staff 
produced a “scan” that synthesized existing 
research and advice from interviews with 
experts. This suggested that the quality of 
feedback is an important factor in new teacher 
development and retention. A Carnegie staffer 
explained: 

While all the key drivers are important 
to address, the one that will likely 
cause changes in other drivers — 
especially at the feedback process 
level — is training coaches, especially 
principals, in the appropriate use of 
the district instructional framework 
and coaching and communication 
strategies. 

The scan informed the development of several 
tools to guide improvement within BTEN. The 
first was the Program Improvement Map, which 
illuminated issues in recruiting, preparing, and 
retaining quality new teachers, as understood 
by network members and reflected in research. 
There is one map for the entire network, which 
was developed jointly by the Aspen Institute, 

AFT, and Carnegie before the districts became 
involved. The second tool was a Fishbone or 
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Ishikawa Diagram, a template for identifying 
the root causes of a specific problem. At one 
end was a clear problem (e.g., many new 
teachers leave the profession). Root causes of 
that broader problem are written at the end of 
what looks like the bones of a fish. There was 
a diagram for the whole network, as well as 
individual diagrams for each of the districts. The 
third tool was a Driver Diagram, in which teams 
specified the critical drivers of change toward 
a desired end. According to a Carnegie leader, 
Driver Diagrams play an ongoing role in “guiding 
improvement research, coordinating inquiry 
across the network, and providing for spread of 
warranted improvements.” The Fishbone Diagram 
is intended to feed into the development of the 
Driver Diagram; the drivers should address the 
most salient of the root causes identified in the 
Fishbone. As with the Fishbone, there was a 
network-level and a district diagram. As districts 
engaged in small tests of change, both diagrams 
were refined to reflect what was learned. 

In BTEN, the network-level Driver Diagram 
identified several clusters of drivers, including 
effective recruitment and placement, professional 
development and support in the placement, 
and assessment and evaluation that support 
enhanced performance (see example at back). 

The districts have begun to engage in the Plan, 
Do, Study, Act (PDSA) cycles that are a hallmark 
of improvement research . 3 In the Plan phase, the 
team made predictions about how many steps 
would be executed with fidelity and which steps, 
if any, they anticipated the principal having to 
modify during the feedback interaction. In the 
Do phase, the principal carried out the protocol, 
noting what happened during the interaction. 
During Study, the improvement team compared 
observations from the Do phase with initial 
predictions made in the Plan phase. Observations 
that confirmed certain predictions helped build 
confidence around the protocol tested; those 
that differed from the predictions (as well as 
other unexpected observations) served as the 
basis for learning and subsequent changes to be 
tested. Based on the analysis, the team decided, 
in the Act phase, to refine steps of the protocol 
and have the principal test the revised version 
through another PDSA cycle. After a few more 
PDSAs and further refinement of the protocol, 
the team had enough confidence in its efficacy to 
have other administrators try it, invoking another 
set of routines aimed at helping bring changes to 
scale across the system. 


Consistent with the I H I model, the changes 
tested and described above were small. One 
Carnegie researcher said that they were “small 
in terms of the intellectual scope of the change, 
but also very small in terms of the sample set in 
which you initially test it.” The premise is that 
these local interactions add up to a positive or 
negative relationship between a new teacher and 
a principal in ways that are consequential for the 
teacher’s induction and development. According 
to a staff member at Carnegie: 

The emphasis is on the "small.” Try 
with one person in one place, a couple 
of times, to learn whether it can be 
warranted as an improvement. Then 
take it up in five places, to study and 
learn the impact of context. Once 
you know that, and have tailored it 
and differentiated into a collection of 
change ideas, expand [to] 25 places, 
study how to make it a permanent 
change. Then, take it to "spread.” 

In the districts, the bulk of the work — conducting 
small tests of change and collecting data — was 
shouldered by local staff, though the work was 
organized slightly differently across districts. 

In one district, a single principal conducted 
most of the tests himself and collected data on 
their effects for the first year of the initiative. 

Fie is now engaging the assistant principal and 
coaches at the school to support his efforts. 

The principal obtained some assistance from his 
district research office in analyzing data, and the 
research office recently fielded a survey to his 
new teachers. In another district, which focused 
on a comparative study of how different schools 
used common planning time for teachers, data 
collection was conducted by an improvement 
team, which included someone from one of the 
institutional partners (AFT). Staff from Carnegie 
and IHI assisted by facilitating the process. 

Challenges 

The NIC approach to improvement was new to 
most of the participants. It requires practitioners 
to adopt a new identity, that of “improvement 
researcher,” which has proven challenging. 

School and district leaders have many other job 
responsibilities besides prototyping improvement 
ideas and collecting data for the network. As a 
consequence, data have not been not collected 
or aggregated as often as intended within the 
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improvement model. The district-level leaders 
and Carnegie both acknowledged the difficulty 
presented by the addition of new responsibilities. 

It also requires a leap of faith to believe small 
tests of change will lead to big, systemic 
improvements. On the Carnegie side, the 
team is pondering “How can small, or enough 
small, become big?” For one district leader, the 
challenge is maintaining enough political support 
for a project that takes a lot of his time, but has 
such small reach so far. Carnegie sees this design 
engineering approach as providing protection 
for innovation, while the practitioner feels 
vulnerable because of the amount of time she 
must devote to the project meetings. 

Benefits 

Even with these challenges, district personnel 
in BTEN find the work a rewarding professional 
learning experience. For example, one district 
leader said BTEN was “hugely valuable because 
we’re doing so much work around measurement 
and thinking about assessing teacher 


performance, and what that looks like, how to 
use it.” 

Another benefit is the attention the process 
gives to implementation, and what Carnegie 
describes as the “challenge of implementing 
good ideas reliably and at scale.” One district 
leader appreciated that researchers were taking 
up and responding to “real-world implementation 
issues” from people who are “building systems 
and wanting tools.” Another mentioned that the 
improvement process requires a careful mapping 
of “what’s happening in schools” and thinking 
through issues of implementation. Both of 
these leaders saw the process as building their 
own capacity for examining implementation of 
reforms in their school systems, a benefit that 
could extend beyond the immediate work on 
BTEN. 

For more information on the Carnegie 
Foundation for the Advancement of Teaching and 
the Building a Teaching Effectiveness Network, 
see: http://www.carnegiefoundation.org/ 


Endnotes 

1 . Anthony S. Bryk, “Support a Science of Performance Improvement,” Phi Delta Kappan 90, no. 8 
(2009): 598. 

2. “Building a Teaching Effectiveness Network,” http://www.aspeninstitute.org/policy-work/ 
education-society/learning-networks/bten. 

3. William Edwards Deming, Out of the Crisis (Cambridge, MA: MIT Press, 1 986) 
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BTEN DRIVER DIAGRAM Part I (Draft 5-29-12) 
Increase new teacher retention and effectiveness 
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